Initialization, Parameters Extraction and Evaluation

This thesis covers topics relevant to model-based coding. Model-based coding is a promising very low bit rate video coding technique. The idea behind model-based coding is to parameterize a talking head and to extract and transmit the parameters describing facial movements. At the receiver, the parameters are used to reconstruct the talking head. Since only high-level animation parameters are transmitted, very high compression can be achieved with this coding scheme. In this thesis, we focus on problems related to initialization, parameters extraction and performance evaluation of face tracking system. The thesis consists of mainly the following aspects as numbered: The initialization problem of model based coding is to fit a generic face model onto a target face in the first video frame. It is the first step toward subsequent motion estimation. 1) We proposed a pseudo-automatic initialization scheme, a Analysis-by-Synthesis based on Simulated Annealing is adopt for the initialization task and proved to be efficient. For parameters tracking, 2) A new scheme of performing texture mapping and motion estimation is suggested which use sample based direct texture mapping. 3) The feasibility of using active motion estimation is explored in Model-based coding which proved to be able to give more than 10 times tracking resolution for rotation. 4) Dynamic Programming is used to solve the “loss of tracking” problem in 2D face tracking. An important problem in Model-based coding is the evaluation of motion parameters. 5) We studied the evaluation problems with the commonly used method which employ a magnetic sensor as “ground truth” and pointed out the problems with such method and the importance of solving the performance evaluation problem.